A Complex Systems Perspective of Risk Mitigation and Modeling in Development and Acquisition Programs

Naval Postgraduate School Acquisition Research Progra

Measuring space systems flexibility : a comprehensive six-element framework

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Aeronautics and Astronautics, 2005.This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections.Includes bibliographical references.Space systems are extremely delicate and costly engineering artifacts that take a long time to design, manufacture, and launch into space and after they are launched, there is limited access to them. Millions of dollars of space systems assets lost annually, when the space system has failed to meet new market conditions, cannot adapt to new applications, its technology becomes obsolete or when it cannot cope with changes in the environment it operates in. Some senior leaders have called for more flexible space systems. The existence of flexibility can help it adapt itself to the change at hand, or even take advantage of new possibilities while in space. Yet in the absence of a practical way to measure its value, most decision-makers overlook its implementation in their space systems. Although the literature is not lacking in number of flexibility measures, there is a void in articulating a unified and comprehensive framework for measuring the multiple aspects of flexibility in space systems. This research is an effort to provide such a framework based on the common fundamental elements that define the nature of flexibility in space systems and other engineering systems. Through the extraction of common elements of flexibility from 25 major papers in the field of space systems flexibility,(cont.) more than 60 papers in the field of manufacturing flexibility and 43 papers in the field of systems engineering, this dissertation identified uncertainty, time window of change, system boundary, response to change, the system aspect to which flexibility is applied, and access to the system as the six key elements that affect the value flexibility. Based on the six elements, the 6E Flexibility Framework was proposed as a twelve-step framework that can guide decision-makers in assessing the value of flexibility in their system. The framework was then applied to four case studies dealing with a variety of space systems (commercial, military and scientific) with monetary and non-monetary value delivery, at different scales (satellite level, fleet level), different time windows of change and with regards to different aspects of flexibility (life extension, instrument upgrade, capacity expansion) facing different kinds of uncertainty (technological change and market uncertainty). The case studies demonstrated the ability of such a framework to provide decision-makers with the information necessary to integrate flexibility in their design and operational decisions and showed that the 6E Flexibility framework could be applied across different aspects of a system easily, capturing the impact of flexibility on design of and decision-making for space systems.by Roshanak Nilchiani.Ph.D

Risk Quantification of Acquisition Programs Through Systems Complexity Measures

Acquisition Research Program Sponsored Report SeriesSponsored Acquisition Research & Technical ReportsThe objective of this research is to mathematically formulate and manage the relationship between the quantitative complexity level of an acquisition or engineering development program and its relationship to the increased technical and programmatic risk, respectively. This research builds upon the PIs previous research experience and grants (NPS BAA 14-002, NPS BAA 15-001, NPS FOA 16001). This research aims to discover and determine the relationship between the quantitative complexity value of an acquisition program (at various points in its lifecycle) as a measure of increased actual technical and programmatic risk respectively. The main goal is to improve the current inaccurate subjective practice of assessment of risk in different stages of a wide range of engineered system development programs as well as acquisition programs. Currently lifecycle risk assessment methodologies such as color-coded risk matrix are heavily subjective in their nature and therefore weak in the assessment of the actual risk. As a result, acquisition programs frequently are exposed to unforeseen technical and programmatic risks and failures; cost and schedule overruns that are due to inaccurate risk identification and assessments. This research proposal focuses on expanding and examining the novel set of new complexity measures that are recently created by our team (with the PIs previous NPS research grants) as pre-indicators of emergence of risks at different stages of a systems development process and lifecycle. The detailed set of created complexity measures, will be modified and categorized based on their application category in physical/hardware/software systems as well as DoD System of Systems level studies. The refinement and categorization of the complexity/risk measures will be applied to and examine several historical case studies of engineered systems success or failures. The focus of this part of research will be on discovering the suitability of each of the 12 complexity/risk measures for application to the right type and category of subsystem/system/SoS of acquisition programs or complex engineered systems. The focus of the case studies chosen will be at refinement and choice of complexity/risk metric to appropriately fit a particular complex engineered system to various manifestation of increased (or decreased) technical as well as some programmatic risks. Multiple historical and theoretical cases of design of complex engineered systems will be studied. The results of this research project will have a broad public purpose in systems development community in various domains of engineering by improving the quantitative assessment of risk from the preliminary and critical design phase, manufacturing and testing, implementation, operation and the retirement of the system. The research result is expected to be applied to a variety of cyber-physical systems as well as DoD systems of Systems (SoS). The complexity-based risk assessment can be applied to various domains of applications such as telecommunication satellite design, regional power infrastructure design and operation, and the next generation of human spaceflight vehicle and many more. The suggested improved methodology can warn the program manager and the other stakeholders on assessing the alternative courses of action at each stage in systems lifecycle as well as reduction and management of the complexity content to mitigating some of the technical risk that a system is facing.Approved for public release; distribution is unlimited.Approved for public release; distribution is unlimited

Tradespace and Affordability – Phase 1

One of the key elements of the SERC’s research strategy is transforming the practice of systems engineering – “SE Transformation.” The Grand Challenge goal for SE Transformation is to transform the DoD community’s current systems engineering and management methods, processes, and tools (MPTs) and practices away from sequential, single stovepipe system, hardware-first, outside-in, document-driven, point-solution, acquisition-oriented approaches; and toward concurrent, portfolio and enterprise-oriented, hardware-software-human engineered, balanced outside-in and inside-out, model-driven, set-based, full life cycle approaches.This material is based upon work supported, in whole or in part, by the U.S. Department of Defense through the Office of the Assistant Secretary of Defense for Research and Engineering (ASD(R&E)) under Contract H98230-08- D-0171 (Task Order 0031, RT 046).This material is based upon work supported, in whole or in part, by the U.S. Department of Defense through the Office of the Assistant Secretary of Defense for Research and Engineering (ASD(R&E)) under Contract H98230-08- D-0171 (Task Order 0031, RT 046)

A Systems Complexity-based Assessment of Risk in Acquisition and Development Programs

Development and acquisition efforts of cyberphysical systems can often encounter cost or schedule overruns due to the complexity of the system. It has been shown that certain amount of system complexity is related to the system functionalities (effective complexity), whereas excessive complexity is related to unnecessary intricacies in the design (apparent complexity). While the former is necessary, the latter can be removed through precise local redesign. One of the major challenges of systems engineering today is the development of tools, quantitative measures, and models for the identification of apparent complexity within the system. This research has the goal of evaluating and measuring the structural complexity of the engineered system, and does it through the analysis of its graph representation. The concepts of graph energy and other spectral invariant quantities allow for the definition of an innovative complexity metric. This metric can be applied knowing the design of the system, to understand which areas are more in need of redesign so that the apparent complexity can be reduced.Naval Postgraduate School Acquisition Research Progra

A Complex Systems Perspective of Risk Mitigation and Modeling in Development and Acquisition Programs

The current methodologies used in risk assessment are heavily subjective and inaccurate in various lifecycle phases of complex engineered systems. The increase in complexity has caused a paradigm shift from root cause analysis to the search of a set of concurrent causes for each event and the relevant complexity content of the system. Many of the systems lifecycle risks are currently assessed subjectively by imprecise methodologies such as color-coded risk matrix and subsequently they suffer from unforeseen failures as well as cost and schedule overruns. This research project proposes a novel approach to major improvement of risk assessment by creating a set of appropriate complexity measures (informed by historical case studies) as pre-indicators of emergence of risks at different stages of a systems development process, and also a framework that enables the decision makers on assessing the actual risk level at each phase of the development based on requirements, design decisions and alternatives. The goals of this research is to capture the complexity of the system with some innovative metrics, thus allowing for better decision making in architecture and design selections.Naval Postgraduate School Acquisition Research Progra

A Systems Complexity-Based Assessment of Risk in Acquisition and Development Programs

Development and acquisition efforts of cyberphysical systems can often encounter cost or schedule overruns due to the complexity of the system. It has been shown that a certain amount of system complexity is related to the system functionalities (effective complexity), whereas excessive complexity is related to unnecessary intricacies in the design (apparent complexity). While the former is necessary, the latter can be removed through precise local redesign. One of the major challenges of systems engineering today is the development of tools, quantitative measures, and models for the identification of apparent complexity within the system. This research has the goal of evaluating and measuring the structural complexity of the engineered system, and does it through the analysis of its graph representation. The concepts of graph energy and other spectral invariant quantities allow for the definition of an innovative complexity metric. This metric can be applied knowing the design of the system, to understand which areas are more in need of redesign so that the apparent complexity can be reduced.Naval Postgraduate School Acquisition Research Progra

Structural Complexity Analysis to Evaluate Technical Risk in Defense Acquisition

The study of "-ilities" in systems engineering has been fundamentally connected to the evaluation of system complexity in recent years. Complexity has been inherent in all defense acquisition programs where technology and human organizations interface. Complexity can be inherent in design of a defense system/System-of-systems, at the organizational layers of defense systems, and in the environment, every now and then imposing its unpredictability or non-linearity to an acquisition program. Increased knowledge and understanding of defense systems complexity can shed light on some various unknown and emergent behavior of such systems, as well as guiding us to better solution sets when facing major decisions or challenges. The goal of our research is to identify, formulate, and model complexity in technical segment of defense acquisition programs, as the increased level of complexity contributes to increased fragility and potential failure of the system. In another word, complexity measure is an indirect measure of risk in complex systems. The future direction of our research aims at replacing a large portion of subject matter experts' opinions on technical systems risk assessment, with actual complex risk measures and therefore improve the decision-making process more objective.Prepared for the Naval Postgraduate School, Monterey, CA 93943.Naval Postgraduate SchoolApproved for public release; distribution is unlimited.Approved for public release; distribution is unlimited